Time-frequency or time-scale representation fission and fusion rules

Time-frequency representations are important for the analysis of time series. We have developed an online timeseries analysis system and equipped it to reliably handle re-alignment in the time-frequency plane. The system can deal with issues like invalid regions in time-frequency representations and discontinuities in data transmissions, making it suitable for on-line processing in real-world situations. In retrospect the whole problem can be considered to be a generalization of ideas present in overlap-and-add filtering, but then for time-frequency representations and including the calculation of non-causal features. Here we present our design for time-frequency representation fission and fusion rules. We present these rules in the context of two typical use cases, which facilitate understanding of the underlying choices. Topics: time-series, time-frequency representations, time-scale representations, computational auditory scene analysis, feature fusion, data fusion, common representational format, libSoundAnnotator.